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Abstract 

Our goal is to find a necessary and sufficient condition on the consensus over a random network, 
generated by i.i.d. stochastic matrices. We show that the consensus problem in three different convergence 
modes (almost surely, in probability, and in L^) are equivalent, thus have the same necessary and 
sufficient condition. We obtain the necessary and sufficient condition through the stability in a projected 
subspace. 
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I. Introduction 
We consider a stocliastic linear difference equation, 

X{t) = A{t)X{t-l), t = l,2,... 

where the states {X{t)} is an M^-valued sequence, and {A{t)} is a sequence of i.i.d. (inde- 
pendent and identically distributed) right stochastic matrices (non-negative matrix with each row 
summing to 1). The system is said to reach consensus if, for any initial state, maxi<j j<7v \Xi{t) — 
Xj(t)\ converges to zero as t — ^ cxo in an appropriate sense. Since X{t) is random, there 
are different modes of consensus: Almost surely consensus, in probability consensus, and 
consensus. 
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Consensus problem over a difference equation has wide applications in random network theory 
(e.g., [[3]|, [HI and the references therein). |l3l studies the consensus in probability. [4] establish 
an elegant necessary and sufficient condition for almost surely consensus by investigating the 
ergodicity of a random matrix sequence: 



where E[-] is the expectation operator and A2(-) is the second largest eigenvalue (in absolute 
value) of the argument matrix. Note that, almost surely consensus implies in probability con- 
sensus, and thus (11.11) is obviously a sufficient condition for in probability consensus. 

In this work, looking further into the specific nature of {X{t)}, we show that the consensus 
in all three modes are actually indifferent, hence (ITTI) gives necessary and sufficient condition 
for consensus in all three modes. In addition, by using a completely different methodology in 
contrast to Q, our result applies to a more general setting: their restriction on the space of 
stochastic matrices with strictly positive diagonal entries can be relaxed (see Remark Hi. II) . 

The main ingredient of our work is that, based on the observation of a relation between 
consensus and stability, the original consensus problem on a sequence is reduced to the stability 
problem on a projected sequence in a subspace. As a result, we can focus our study on the 
eigenspace structure of the projection operator. As a by-product, we offer a simple proof of 
consensus to deterministic linear networks. 

The rest of the paper is arranged as follows: We start with the problem formulation in section 

2, where a crucial result on the relation between consensus and stability is presented. In section 

3, a simple proof of consensus on a deterministic sequence is provided, which can be read 
independently for readers only interested in the deterministic case. The main result, the necessary 
and sufficient condition for consensus of a random network, is established in section 4. Finally, 
we conclude our investigation in section 5. 



In the first subsection, a crucial result (Theorem III. II) on the equivalence of consensus and 
stability in a subspace will be presented under a general setup of consensus problem. This 
theorem can be applied to very general cases setup, including nonlinear and random sequences, 
and plays an important role throughout the paper. In the second subsection, the main consensus 
problem is formulated using a linear stochastic difference equation. 




(I.l) 



II. Problem formulation 
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Before proceeding, let us recall some standard notations: 

1) In (column) vector space R^, Xi is the ith coordinate of vector x G M^; /^-norm is 
ll^llp = J2iLi kil^' VI < p < oo; denotes the transpose of x. 

2) In square real matrix space R^^^, / is the identity matrix; for all A e M^^^, ||^||p ~ 
max||j;||=i for all 1 < p < oo; the eigenvalues will be arranged in order of |Ai(^)| > 
|A2(A)| > • • • > |Ajv(^)|; the spectral radius refers to p{A) ^ \Xi{A)\. 

3) II • II is used in the formula if it is valid for all -norms. 

4) Given a probability space P), we denote by E = E"" the expectation under P. 

LP refers to LP{n,T,F): for random vector Y : Q ^ R^, the L^-norm is ||F||lp = 



A. A general consensus problem 

Let (Q, J^, P, F) be a filtered probability space, where F — {Tt : t 0, 1, 2, . . .} is a sequence 

of increasing ci-algebras with J^^o C J-. We consider an F-adapted sequence {X(t)} taking 
values in M^. In other words, X{t) is a measurable mapping from {^l,J-'t) (M^, i5(]R^)), 
where fi(M^) is the Borel cr-algebra on M^. Such a sequence includes the general form of 



for some measurable function ft, and emphasizes its independence of future events First, we 
start from the precise definition of consensus on random sequence in three different modes. As 
usual, X{t,uj) will be used instead of X{t) when we need to emphasize its dependence on a 
sample path a; e il. 

Definition II.l (Consensus of a sequence). Let {X{t)} be an ¥ -adapted -valued random 
sequence. {X{t)} is said to reach consensus 
1) in probability, if 



(/J|FH||#(d^))Vp = (E[(||F||2F])Vp. 



X{t) = ft{X{t-l),...,X{l)) 



2) almost surely [with probability 1), if 




Xi{t,Uj)-Xj{t,Uj)\ >£} 



= 0, V£ > 0. 




1. 
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3) in If {p> 1), if 



lim E 



ma^ |X,(t)-X,(t)P 

l<t,j<N 



Definition II.2 (Stability of a sequence). {X{t)} is said to be stable {at zero) 

1) in probability, if 

lim Pjo; e : ||X(t,cc;)|| > = 0, > 0. 

2) almost surely, if 

f\uj e Q : lim = o| = 1. 

3) in LP {p > 1), if 



lim E 

t—>-oo 



0. 



\X{t,uj)\\ 

Define a subspace of IR^ by IRq = {^^ e : xi — X2 — ■ ■ ■ — xn}- Let 11 be a projection 
operator on Wq, i.e. 

Ux = {x,Vo)vo, Vx e Rq, 

where (•, •) is inner product, Vq G IRq is an Z^-norm unit vector. We thus have the orthogonal 
projection n-*- : — > IR-*- by n-*- = 7 — 11, so that the orthogonal decomposition is valid 

x = Ux + U^x, Vx e M^. (II. 1) 

The following theorem shows that the consensus of a sequence in is equivalent to the stability 
of the sequence projected on the subspace Mq . 

Tlieorem II.1. {X{t)} reaches consensus almost surely {respectively, in probability, or in LP) 
if and only if {YV-X{t)} is stable almost surely {respectively, in probability, or in L^). 

Proof: We will show the equivalence of stability and consensus in the sense of almost 
surely. The equivalence in probability and in can be similarly proved. 
(=>) Suppose {X{t)} reaches consensus almost surely. Define Y{t) e Rq be a vector with all en- 
tries equal to the value of first coordinate of X{t), i.e. Y{t) = {Xi{t),Xi{t), . . . , Xi{t))'^. 
Since 

||n-LX(t)||oo = miiij^gB;,, \\X{t) - ylloo 

< ||x(t)-r(t)|U 

= maxi<j<Ar \Xi{t) - Xi{t)\ 

< maxi<jj<jv \Xi{t) - Xj{t)\ 
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as t — 7- oo almost surely. Therefore, {n^X(t)} is stable almost surely. 
(<^=) Suppose {n-'-X(t)} is stable almost surely. Let (nX)j(t) is the zth coordinate of vector 
nX(t). Note that, since IlX{t) E Mq, we have all coordinates with the same value, that 
is, (nX)j(t) = (JlX)j{t),\/i,j. Therefore, by triangle inequality, 

max,,, \X,{t) - X^{t)\ < max,,,(|X,(t) - (nX),(t)| + |(nX),(t) - X^{t)\) 

< maxi \Xi{t) - {UX)i{t)\ + max,- \(nX)j{t) - Xj{t)\ 

<2||x(t)-nx(t)|U 
= 2||n^x(t)|U^o 

as t — )■ oo almost surely. Therefore, {X{t)} reaches consensus almost surely. 



B. Consensus problem over linear random network 

We consider a similar setting as flU. Let the space of N x N stochastic matrices be 

N 

Sn = ^A = {aij)NxN ■ o-ij > 0, ^ajj = l,Vz,j| (II.2) 

and B{Sn) be the Borel a-algebra on Sn- Let /i be a given probability distribution on {Sn, B{Sn)), 
and {A(t)} be an ^Ar-valued i.i.d. sequence with distribution /x. Let = (Sn)"^, P = i^x jj,x - ■ ■ , 
and J-Q = {0, J^t = (r{A{l), A{2), . . . , A{t)) for t > 1, F = U^oJ^t- 
Now, we consider a random sequence {X{t)} given by 

X{t) = A{t)X{t - 1), Vt G N; X(0) = x. (113) 

Then, {X{t)} is an F-adapted sequence in the filtered probability space J-", P, F). Observe 
that the distribution of X(t) is determined by the initial state X(0) = x and distribution fi. 
Sometimes, we write X^(t) to emphasize the initial state X(0) = x in the context. 

Definition II.3 (Consensus and stability of a distribution). A distribution ji is said to reach 
consensus almost surely {respectively, in probability, or in L^), if {X^{t)} of (in.3l) generated 
by the distribution jj, reaches consensus almost surely {respectively, in probability, or in L^) for 
all initial states x G M^. 

Similar to Definition IIL3[ one can define stability for the sequence (|IL3I) generated by the 
distribution /i. 
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Remark II.l. [4J had a different problem formulation in that the space Sn of (IIL2I ) was replaced 
by a smaller space 

Sn = {A e Sn '■ all diagonal entries are strictly positive }. 

Indeed, such a restriction is crucial in the proof of [4, Theorem 3] to utilize the [1, Perron- 
Frobenius theorem] on primitive matrix. In our work, the diagonal entries can be zero, which 
therefore covers the results of yj as a special case with fi{SN \ Sn) = 0. □ 

Next, our goal is to find a necessary and sufficient condition for the consensus of the distri- 
bution yU. 

III. Necessary and sufficient condition for a deterministic sequence 

A deterministic system can be treated as a special case of a random system in the following 
sense. Let the probability distribution /i on Sn satisfy /i({y4}) = 1 for some stochastic matrix 
A E Sn- Then, A{t) = A for all t = 1,2,..., and the sequence {X(t)} of (III.3I) becomes 
deterministic, and in this case we have 

X{t) = A^x, Vt = 0,l,2,... 

For convenience, we say A reaches consensus if {X{t) = A^x} reaches consensus for all 
initial states x E M^. Note that, the deterministic consensus is indifferent to all three modes of 
consensus, since the sample space can be treated as a singleton {A} x {A} x ■ ■ ■ . Thus, this 
definition is consistent with Definition III.3I on consensus (in all three modes) of distribution fi 
of the form fi{{A}) = 1. 

The main idea in this section is that, thanks to Theorem lll.li it is equivalent to find a sufficient 
and necessary condition of the stability of {Il-^X(t)}, which turns out to be a sequence generated 
by the projection matrix U^A. We will show that U^A is stable if and only if p{Il^A) < 1 by 
Proposition IIII.2[ Together with the fact that p{Il-^A) = \\2{A)\, by Proposition llll.li we will 
obtain the desired necessary and sufficient condition. 

To proceed with the consensus on the deterministic sequence X{t) generated by the stochastic 
matrix A, we first recall some properties of stochastic matrices. Since each row sum of a 
stochastic matrix is equal to 1, its largest eigenvalue is p{A) = \i{A) = 1, i.e. Ax = x 
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for all X G Mq. Also, we have 



P*x||oo < \\x\\oo, Vx G M^,t G N 
In addition, we have the following useful results: 

Proposition III.l. Let A G Sn be a stochastic matrix. Then 
1) A has a Jordan canonical form of 

1 Oix(Ar-i) 



(111.1) 



A 







(Af-l)xl A22 



(111.2) 



where Omxn is m x n matrix with each entry being zero, and A22 is sub-matrix of Jordan 
form. 

2) The linear operator n-*- defined in (III. II) satisfies 



3) H A, as a matrix, has a Jordan form of Aq 
particular, piJl^A) = \\2{A)\. 



U^A = U^AU^ 




A 



22 



(III.3) 

with A22 defined in (|in.2l) . In 



Proof: 

1) Let vq be the unit vector in the subspace Rq. Note that vq G Mq is an eigenvector of A 
associated with the eigenvalue \i{A) = 1, i.e. (A — I)vq = 0. To prove the first claim, 
we only need to show that the Jordan block corresponding to \i{A) = 1 is simple. If not, 

there exists vi ^ Rq associated with the Jordan block 



1 1 
1 



satisfying 



{A - I)vi = vo. 



(111.4) 



By induction. 



A^vi = t-vo + vi, Vt G N. 



This implies that HA^tiiHoo — )■ 00 as t — )■ cx3, which leads to a contradiction to (IIII.ll) . 
2) One can prove (IIII.3I ) as follows: Wx G 



U^Ax = U^AiJlx + U^x) = U^AUx + U^AU^x = U^Ux + U^AU^x = U^AU^x. 
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3) If {xi, . . . Xm} are generalized eigenvectors of A associated with some eigenvalue A in the 
Jordan block in A22, satisfying 

{A — XI)xi = for z = 1, 2, . . . m, xq = 0, 

then, by the facts xi ^ Mq and (|in.3D . 

{u^A- xi)(n^Xi) =u^An^Xi- xu^Xi 

= U^Axi - XU^Xi 
= U^{A-XI)xi 
= U^Xi.i. 

In other words, since II-'-xi 7^ 0, U-^A with {U-^xi, . . . ,11-^Xm} preserves the structure 
of the eigenspace associated with matrix A corresponding to the eigenvalue A in Jordan 
block A22- Also, since A is arbitrary eignevalue in the Jordan block A22, together with 

U^Avo = 0, we conclude Il-'-yl has a Jordan form of Aq = 



. Finally, we have 





A22 

p(n^A)=p(Ao) = p(A22) = |A2(A)|. 

■ 

Next, we present a necessary and sufficient condition for stability. 

Proposition IIL2. A e R^""^ is stable if and only if p{A) < 1. 

Proof: One can use the fact lim^^^oo 11^*11^^* = p(^) to complete the proof. ■ 
Thanks to Proposition IIII.ll and Proposition IIII.2[ we are now ready to obtain a necessary and 
sufficient condition for the consensus of a deterministic sequence. 

Theorem III.l (Necessary and sufficient condition in deterministic case). A E Sn reaches 
consensus if and only if \X2{A)\ < 1. 

Proof: By Theorem lll.li A reaches consensus if and only if {I[-^X{t)} is stable. Note that, 
by (IIII.3I) . for any initial state x G 

n^x(t) = ii^AX{t - 1) = (n^A)n^x(t - 1) 

= • • • = ili^Ayii^x = {U^AYx. 
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Thus, {n^X(t)} is a sequence generated by H-^A. By Proposition IIIL21 {]l-^X(t)} is stable if 
and only if p{Il-^A) < 1. Observe that, by Proposition IIII.ll p{Il-^A) = \X2{A)\. This completes 
the proof. ■ 

IV. Necessary and sufficient condition for a stochastic sequence 

In this section, we return to the stochastic sequence {X(t)} defined in (|II.3I) generated by 
distribution /i, and study a necessary and sufficient condition for its consensus. First, by studying 
the fine structure of the random sequence generated by i.i.d. stochastic matrices, we show that 
consensus in three different modes classified by Definition III. II are in fact equivalent to each 
other. Thus, we can only work on the almost surely consensus. 

A. Equivalence of consensus in three modes 

Before we proceed with the equivalence of consensus in three modes, we briefly recall some 
relations between convergence of random variables in three modes, and we refer to [2] for 
more detail. Consider a sequence of random variables {a„, n = 1, 2, . . .} and a random variable 
a > 0. Both almost surely convergence and convergence imply in probability convergence, 
i.e. a„ — 7- a almost surely implies a„ — t- a in probability; a„ — )■ a in implies a„ — )• a 
in probability. However, the reverse directions need further conditions in general. a„ — )■ a in 
probability together with |a„| < |6| almost surely for some b E J^, P) implies a„ — a in 

by the dominated convergence theorem; a„ — )■ a in and < a„_|_i < almost surely 
together implies a„ — )■ a almost surely by the monotone convergence theorem. 

Lemma IV.l. Consider the sequence {X(t)} defined in (III. 31) generated by distribution /i. Given 
X{0) = X, the following statements on stability of {Il^X(t)} are equivalent: 

1) {U^Xit)} is stable in probability. 

2) {U^X{t)} is stable in L\ 

3) {U^Xit)} is stable almost surely. 

Proof: Observe that, by (|III.3I) . {Il-^X{t)} is a sequence generated by random matrix 
U^A{t), i.e. 

n^X(t) = U^A{t) U^X{t - 1). (IV.l) 
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In the following, we prove the equivalence by showing: (1) implies (2), (2) implies (3), (3) 
implies (1), respectively. 

1) If the sequence {]l^X(t)} is stable in probability, then ||n^X(t)||oo — )• in probability. 
Together with the uniform boundedness ||n-'-X(i(:)||oo < ||a^||oo» the dominated convergence 
theorem implies that ||n-'-X(t) ||oo — in L^. Thus, the sequence {Il-^X{t)} is stable in 
L\ 

2) If the sequence {U-^X(t)} is stable in L\ then ||n-LX(t)||oo in L^. In addition, one 
can show the monotonicity of ||n-'-X(t) ||oo < ||n-'-X(i(: — l)||oo, by observing 

||n^x(t)||oo = \\ii^A{t)x{t - i)|U = ||n^A(t)n^x(t - i)||oo < ||n^x(t - 

(IV.2) 

By the monotone convergence theorem, ||n-'-X(t) ||oo — ?■ almost surely. 

3) It is well known that almost surely convergence implies convergence in probability. 

■ 

The next theorem about equivalent consensus in three modes is a main result in our paper. 

Theorem IV.l. Consider the sequence {X(t)} defined in (111.31 ) generated by distribution fi. The 
following statements on consensus are equivalent: 

1) Distribution fi reaches consensus in probability. 

2) Distribution fi reaches consensus in L}. 

3) Distribution /i reaches consensus almost surely. 

Proof: It follows from Theorem III. II and Lemma IIV.II ■ 

B. Necessary and sufficient condition for the random case 

Thanks to Theorem IIV.II our work is now reduced to finding a necessary and sufficient 
condition for consensus in any one of the three modes. Below, we say the distribution /i reaches 
consensus without specifying a convergence mode. 

Recall from Definition III. 31 that, a distribution /i reaches consensus if the generated sequence 
{X^(t)} reaches consensus for all initial states x E M^. The next proposition shows that it is 
sufficient to check the consensus of {X^{t)} only for all x G (M+)^ to guarantee the consensus 
of a distribution /i. 



April 22, 2010 



DRAFT 



11 

Proposition IV.l. fi reaches consensus if and only if X^(t) defined in (IIL3I) reaches consensus 
for all X G (M+)^. 

Proof: Observe that = + c for all c G Mq- Hence, 

max |Xf (t, oo) - X^t, uj)\ = max |Xf+"(t, u) - X^+'{t, uj)\, Vc G Mq- 

l<i,j<N ■' ^<id<N' ■' 

In other words, to consider consensus of {X^{t)} for some x ^ (]R+)^, one can always 
investigate the consensus of X^+'^(t) equivalently, by taking c = (||x||oo, • • • , ||a;||oo)^ ^ I^o- 
Note x + c G (M^)^, hence the result holds. ■ 
Next, we review some useful properties of the expectation operator E. First, the expectation 
operator E is commutative with any deterministic matrix A, i.e., 

AE[F] = E[AF], VJ^-measurable Y -.Vl^M.^. (IV.3) 

In particular, by taking A = H^, we have II^E = EII-'-. Furthermore, for an arbitrary random 
matrix A, if a random vector F : — )• is independent of A, then 

E[AY] = E[A]E[Y]. (IV.4) 

Finally, note that the deterministic sequence {E[X(t)]} is actually a sequence generated by the 
deterministic matrix E[A], since by (|IV.4I) 

E[X(t)] = E[A(t)X(t - 1)] 

= E[A{t)]E[X{t - 1)] (IV.5) 
= E[A{l)]-E[X{t-l)]. 

Theorem IV.2 (Necessary and sufficient condition for consensus). Consider {X(t)} defined in 
(III. 31) generated by distribution /i. /i reaches consensus almost surely (also, in probability, and 
in L^) if and only A2(E^[A(1)]) < 1. 

Proof: By saying that fi reaches consensus, we mean fi reaches consensus in any of three 
modes due to Theorem IIV. 1 1 

{=^) If /i reaches consensus, then Theorem III. 1 1 implies Il-^X{t) — )■ in for all initial states 
X(0) = X, hence E[||n^X(t)||oo] 0. 
Next, by (|IV.3I) and Jensen's inequality, we have 

||n^E[x(t)]|U = ||E[n^x(t)]|U < E[||n^x(t)|U] ^ o 
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This implies that the deterministic sequence {n^E[X(t)]} is stable. Then, using (IIV.5I ) 
and (|nL3l) . we have 

U^E[X{t)] = n^E[A(l)] ■ E[X{t - 1)] = n^E[A(l)] ■ U^E[X{t - 1)] 

In other words, {n-'-E[X(t)]} is a deterministic sequence generated by matrix n-'-E[y4(l)]. 
Thus, by Proposition Iml] and Proposition HIOl p(n^E[A(l)]) = |A2(E[A(1)])| < 1. 
(<^=) It follows from (IIV.5I) that the deterministic sequence {E[X(t)]} is generated by ma- 
trix E[A(1)]. If |A2(E[A(1)])| < 1, then by applying Theorem Iml] on (HVSl) . we con- 
clude that the sequence {E[X(t)]} reaches consensus. Hence, the deterministic sequence 
{n^E[X(t)] = E[n^X(t)]} is stable by Theorem HIl i.e., E[n^X(t)]||i ^ 0. By 
Proposition llV.li we can always assume x G (M+)^. Thus, n^X(t) G (M"^)^, and this 
leads to 

E[||n^x(t)||oo] < E[||n^x(t)||i] = ||E[n-^x(t)]||i ^ o. (iv.6) 

In other words, {n-'-X(t)} is stable in L^. This implies the consensus of fi by Theorem lll.ll 

■ 

Remark IV.l. In (ITV^ . we used the fact, for aWY -.Q^ (M+)^, 

N N 

E[||y||i] = e[^f,] = J]E[y,] = ||Ey||i. 

i=l i=l 

However, one shall not expect identity E[||F||oo] = ||E[F]||oo holds in general. For instance, one 
has strict a inequality, if Y{uJi) = (0,1)^, ¥{002) = (1,0)^, and P({c^i}) = ^{{002}) = 1/2, 
then E[||F||oo] = 1 > | = ||E[F]||oo holds. This is the reason that we use the /^-norm in (I IV. 61) 
instead of directly but incorrectly using E[||n^X(t)||oo] = ||E[n^X(t)]||oo ^ 0. 

Corollary IV.l. Consider {X(t)} defined in (III.3I) generated by distribution /i. /i reaches con- 
sensus if and only ifE^[A{l)] reaches consensus. 

V. Concluding remarks 

In this paper, we derived a necessary and sufficient condition for consensus over a linear 
random network based on the connection between consensus and stability as shown by Theo- 
rem lll.li Although our proof is shown under the discrete-time framework, the similar results 
still hold in the continuous-time setting. Consequently, one can similarly follow the procedure to 
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obtain consensus conditions based on stability results of ^ on hybrid switching continuous-time 
systems. 

Regarding the second-order random network, one can also utilize the result of this work. More 
precisely, for 

X{t) = aA{t)X{t - 1) + l3B{t)X{t - 2), t = 2, 3, . . . 

where a + /3 = 1, a, /3 > 0, {A{t)} and {B{t)} axe i.i.d. stochastic matrix sequence with given 
distributions fiA and fiB on Sn, the problem is equivalent to 



Y{t) = C{t)Yit-l), 

X{t) 
X(t-l) 



is an M^^-vector and C{t) 



aA{t) l3B{t) 
I 



is a stochastic 



where Y{t) = 
matrix in S2n- 

One last interesting remark is the application of Kolmogorov's 0-1 law [|2l, which is firstly 
given in flU in the context of ergodicity of i.i.d. matrix sequence. Similar result also holds in the 
consensus and the stability problems. For instance, now we know that X^(t) defined in (|II.3I) 
does not reach consensus when A2(IE[A(1)]) > 1 for a given distribution fi in any of the three 
modes. In other words, 

pjw G VI : X^(t, reaches consensus for all x E M^j < 1. 

Natural question is then, what the above probability is when A2(IE[A(1)]) > 1. The answer is 
surprisingly simple: zero. 

Proposition V.l. Consider X^{t) defined in (III. 3 1) generated by i.i.d. matrices with distribution 
fi. Then 

pju; G Q : X^{t,ijj) reaches consensus for all x G 



is either 1 or 0. 

Proof can be accomplished similarly to JH Lemma 1], by using the tail a-field argument on 
decreasing events of the form 

Bk = l^u : Ii'^f.A{t)x reaches consensus for all x G R^|. 
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